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Abstract 

The International Space Station will soon provide 
an unparalleled research facility for studying the near- 
and longer-term effects of microgravity on living 
systems. Using the Space Station Glovebox Facility - a 
compact, fully contained reach-in environment - 
astronauts will conduct technically challenging life 
sciences experiments. Virtual environment technologies 
are being developed at NASA Ames Research Center to 
help realize the scientific potential of this unique 
resource by facilitating the experimental hardware and 
protocol designs and by assisting the astronauts in 
training. The “Virtual GloveboX” (VGX) integrates 
high-fidelity graphics, force -feedback devices and real- 
time computer simulation engines to achieve an 
immersive training environment. Here, we describe the 
prototype VGX system, the distributed processing 
architecture used in the simulation environment, and 
modifications to the visualization pipeline required to 
accommodate the display configuration. 

Introduction 

Onboard the International Space Station, complex 
life science experiments will address long-standing 
questions concerning life’s ability to adapt and respond 
to the space environment. Many of these experiments 
will require astronaut intervention and the use of the Life 
Sciences Glovebox Facility (LSG). Within the LSG, 
astronauts manipulate scientific instruments, conduct 
experimental assays, collect tissue specimens and 
perform delicate dissections — all under highly controlled 


conditions and while physically tethered to minimize 
drift and within strict time constraints. The experiments 
often demand very detailed training and knowledge of 
instrumentation, surgical anatomy and specific scientific 
objectives. To be successful, astronauts must remain 
highly proficient in these experimental techniques, but 
due to scheduling constraints they can receive only 
limited Earth-based training with real LSG mock-ups 
and real experimental specimens. Because of mission 
costs, crew safety, and the immediate and far-reaching 
impact of the scientific results, it is imperative that novel 
strategies be developed to maximize the use of the LSG 
Facility. To this end an immersive virtual environment 
simulation system called the “Virtual GloveboX” (VGX) 
is being developed at NASA Ames Research Center 1,2 . 
Astronauts and support crews can use the VGX for early 
engineering development and experiment planning, and 
later advanced training activities that require the LSG. 
The system integrates off-the-shelf hardware and 
software with new real-time simulation technologies to 
provide a realistic 3-D virtual environment, mimicking 
the real LSG environment. Within the VGX, the force of 
gravity can be turned on and off, allowing astronauts to 
perform experiments in a simulated microgravity 
environment while still on Earth. Further, a continuum 
of scenarios from ideal conditions to catastrophic 
experiment failures, such as contamination, can be 
introduced into the training session to allow the 
astronaut and principal investigator to assess critical 
decisions. The VGX does not eliminate the need for 
direct, hands-on training and experiment planning using 
physical mock-ups; however, it can streamline these 
processes, reduce the need for full-scale training 


sessions with live animals, increase the frequency and 
ease of the principal investigators input to their specific 
experiment, and provide astronauts with a means to keep 
their skills sharp on Earth and in space, thereby 
maximizing the efficiency and success of ground- 
breaking biological experiments in space. 

Virtual GloveboX System (VGX) 

The VGX is composed of a package of virtual 
environment technologies that sets high standards for 
speed, resolution and desktop user-interface immersion. 
Unlike any commercial product or research prototype 
before it, the VGX is designed to combine real-time 
graphics, force-feedback haptics and hand tracking in an 
immersive desktop environment that is registered with 
physical space. The VGX hardware and software were 
assembled specifically for engineering development, 
experimental design and astronaut training for the Space 
Station Life Sciences Glovebox Facility (LSG), but the 
system is also versatile and could be used for other 
virtual environment simulations and training that occur 
in a desktop-sized volume. 

A Stereoscopic Display Station is used to provide 
an immersive virtual environment for the VGX simulation 
software (Figure 1). The Stereoscopic Display Station is 
currently designed to use twin LCD projectors with 
circular polarization options to provide high-resolution 
images (1280 x 1024 pixel resolution) while maintaining a 
compact display structure. The image projected into the 
box falls on a flat 40”x30” screen that is above the user’s 
arms. The user looks down onto a front-projected screen 
in the Display Station, and with the use of lightweight 
circularly polarized stereo glasses a 3D virtual 
environment 40” wide, 30” deep and 24” in height is 
created for user interaction and visually overlaid with the 
physical structure of the VGX enclosure. This internal 
working space of the Stereoscopic Display Station is 
almost identical to the space inside the LSG, which is 
scheduled to fly aboard the International Space Station. In 
contrast to other “reach in” virtual environments that use a 
monitor and a mirror to create the reach in space, the use 
of a front projected screen allows the image to remain in a 
stable position on the display when the user moves their 
viewing point. This stable, full screen image is what 
allows the display to be visually overlaid with the 
glovebox enclosure, increasing the sense of immersion by 
aligning the perceived glovebox walls and floor with the 
real glovebox structure. 

Inside the desktop of the VGX, the user will be 
able to interact with virtual objects through a pair of 
Cybergloves® (Immersion Inc.) that are tracked in the 
virtual environment using magnetic trackers (Ascension 
Technology Inc.). The operator inserts his/her hands 
through the front armholes of the box and into the 


gloves. A virtual representation of the user’s hands are 
shown on the viewing screen in the box and are 
registered with the real position of the user’s hands in 
the working volume. Desktop PHANToM™ force 
feedback devices (SensAble Technologies Inc.) provide 
a three-degree-of-freedom haptic interface to the user. 



Figure 1: A Stereoscopic Display Station uses 
two digital projectors and polarized optics to 
provide a high-resolution immersive 
environment of about the same size as the 
working space of the Life Sciences Glovebox 
Facility. 

Virtual tools in the environment that require force 
feedback for their proper use can be associated with the 
PHANToM device and can be picked up by the user. 
Registration of the virtual environment to the physical 
space within which the user works requires that the 
VGX display be tracked to the position and orientation 
of the users viewpoint (i.e., eyes), restricting the use of 
the VGX to a single operator. However, the distributed 
design of the simulation environment allows for multiple 
users from separate VGX systems to interact in the same 
virtual environment, simulating the multi-participant 
scenarios anticipated for training and complex 
experimental protocols. 

General Architecture 

The simulation and visualization software for the 
VGX is designed around a networked cluster model to 
allow maximum performance and scalability for each 
component of the system. The system consists of three 
major subsystems: 1) A simulation engine to monitor 
and calculate interactions of the objects in the virtual 
environment, 2) peripheral device controllers to allow 
interaction of the user with the virtual environment, and 
3) a display system to visualize the virtual world at 
multiple workstations. Currently, the core simulation 
engine and peripheral device controller is the Spring 



simulation environment developed with the National 
Biocomputation Center at Stanford University 3 . This 
physically based modeling system is coupled with a 
highly efficient collision detection algorithm to allow a 
rich set of 3D objects to interact with one another and 
the user via virtual representations of the peripheral 
devices. Some simple rigid body interactions allow for 
manipulations of virtual laboratory equipment - the user 
can pick up and move objects, practice setup/break down 
procedures, and arrange equipment within the 
environment during feasibility studies. More complex 
interactions can be had through a real-time deformation 
engine used to simulate soft-tissue interactions ( e.g ., 
surgical simulation). The peripheral devices 
(Cybergloves, PHANToMs) communicate with the 
simulation engine via a network interface, allowing each 
device to be controlled by a separate computer without 
regards to proximity to the simulation engine or display 
systems. The orientation and movements of each 
peripheral device are registered with the virtual 
environment, mapping each device to the world 
coordinates of the virtual environment. A specialized 
visualization subsystem was created to accommodate the 
unique architecture of the VGX display and off-load the 
visualization processing from the main simulation 
system. This visualization system consists of a data 
replicator that sends the changing geometric information 
from the simulation engine to the display systems, a 
custom stereo rendering system to address the twin LCD 
projectors, and a post-processing facility within the 
display systems to enhance the geometric data for 
display. 

Display Subsystem 

The stereoscopic display of the VGX dual 
projector system is achieved by creating a virtual display 
space on a single dual output graphics card (using the 
nVidia Quadro4 graphics chipset), implemented in 
OpenGL. This display space is achieved through three 
components: 1) a mapping of the left- and right-eye 

view that can be displayed through the dual output 
system of the graphics board, 2) a geometry replicator 
that receives all 3D geometric data from the simulation 
engine via a network interface, and 3) a viewpoint 
tracking system used to create the viewport mapping for 
each eye to the display. All communications are 
performed over a high-bandwidth network (preferably 
Gigabit Ethernet) to maintain the performance of the 
rendering system. 

The mapping of the left- and right-eye views to 
the dual output of the graphics card requires architecture 
different from that typically used in OpenGL stereo 


display systems. The dual outputs of the graphics card 
are configured to be horizontally concatenated, 
providing a side-by-side display that is (2x1280) x 1024, 
with each output displaying a 1280x1024 portion of the 
display. Rather than using an OpenGL dual buffer 
system that explicitly allows the left-eye viewpoint to be 
drawn in one buffer, and the right-eye in the other, we 
use a single buffer that is twice the width of each view, 
and draw the left viewpoint on one side and the right 
viewpoint on the other. Thus for a display system 
configuration that requires each projector to receive an 
1280x1024 image, the left-eye viewpoint is rendered to 
the display buffer in the range (0-1279,0-1023) and the 
right eye viewpoint is rendered to the display buffer in 
the range (1280-2559,0-1023), and each of the outputs 
on the graphics card shows precisely one eye’s 
viewpoint. The advantage to this method are both that 
the display of the left and right eye images are fully 
synchronized in their display timing which effectively 
genlocks the projectors, and the display can be generated 
from a single PC using a dual output graphics card 
which eliminates the need to coordinate the display 
between different computers for the two different 
projectors. 

The geometry replicator is a high-bandwidth 
connection between the display computers and the 
simulation engine used to move the geometric data from 
the interactive environment to the display environment. 
This is a strict Client/Server model, with information 
flowing from the simulation engine to the display 
systems. Using a Gigabit Ethernet connection, the 
simulation engine can effectively move as much data to 
the display systems as possible sending directly across a 
PCI bus to an internal graphics card. Upon initialization 
of the system, all the geometric data within the 
simulated environment is sent to the display systems to 
create the initial visual environment. Since the optimal 
organization of the geometric data is different for the 
simulation engine and display systems, the display 
systems reorganize the data to map it as efficiently as 
possible to the graphics system available. During 
subsequent modifications of the data during user 
interactions in the simulation environment (e.g., object 
movements, soft tissue deformations, cutting 4, ’ , etc.), 
only the modified geometry is replicated to the display 
systems. Because the simulation and interaction engine 
operates strictly on geometric primitives, it is left to the 
display system to implement how the data should be 
presented to the user. This allows the display to take 
advantage of the features of today’s newest graphics 
cards (multi-pass texture processing, bump mapping, 
vertex and/or pixel shaders) without impacting the 



performance of the simulation itself. This approach 
provides a means for creating a significantly richer 
visual presentation than that required for calculating the 
physical interactions, and allows the display systems to 
take advantage of any specific graphics hardware 
available at each display station participating in the 
simulation. 

The viewpoint-tracking component is required to 
compensate for the users constantly changing position 
and orientation to the display screen. Because the 
location and orientation of the screen relative to the 
user’s eye deviates significantly from the standard 
perspective view, a series of modifications to the 
viewport mapping must be made for each eye to 
generate a visual image that is consistently registered 
with physical space. A typical viewing point would 
place the user at a relative angle to the screen of nearly 
45 degrees, creating substantial distortions in the image 
relative to the standard flat screen model. Projecting a 
line from the user’s eye perpendicular to the viewing 
plane, an asymmetric viewing frustum can be defined to 
match the physical location of the comers of the display 
and passed to the rendering engine, and continually 
updated via the tracking system. Due to the proximity of 
the user to the large viewing surface, a viewing frustum 
unique to each eye must be maintained for proper stereo 
viewing. 

When used with the LCD projectors, the display 
system requires a final adjustment phase to fine tune the 
image alignment between projectors. Because the 
LCD’s lack the internal functions to modify the 
projected image found in high end CRT systems, the 
precise pixel-to-pixel alignment required for effective 
stereo is difficult and time consuming to achieve through 
physical adjustment of the projectors. An efficient 
method of compensating for image misalignment on 
today’s PC graphics hardware is to take advantage of 
off-screen rendering and texturing optimizations. 
Assuming the distortions between images are small, a 
rectangular grid can be projected to fill the viewport, and 
the position of each grid point can be modified slightly 
to correspond between images. Rendering the scene to 
an off-screen buffer and then projecting this image onto 
the grids via the texture buffer provides a high degree of 
alignment, and significantly reduces the work required 
to setup and maintain the display system. 

Distributed Graphics Performance 

The performance of the graphics subsystem is 
strongly dependent on the simulation engine, and the 
speed of data replication that can occur. In the Spring 
simulation system, all of the geometric primitives of an 


object are updated for any modification of the position 
or orientation of an object. This design reflects the 
emphasis on soft-tissue dynamics in the simulator that 
stress modifications of the shape of objects, and 
efficiently packages the data to allow such operations. 
In a typical application in the VGX, several objects with 
a total of -10,000 - 15,000 geometric primitives 
(triangles) will be in motion and replicated to the display 
system. Geometry updates of 15 - 20 frames per second 
are achieved with such a data load, with the graphics 
system providing a rendering rate of greater than 35 fps 
for all objects (stationary and dynamic) in the scene. If 
the projectors are physically aligned to allow an accurate 
stereo overlay on the screen, the effective rendering 
speed is sufficient to allow smooth motion at 35 fps. 
However, a significant performance drop is taken when 
creating the display through the texture buffer, which 
requires moving large amounts of data across the 
graphics bus. We have achieved a consistent 6+ fps 
with this technique for 1280x1024 images, rendering 
each eye view to the texture buffer and then applying the 
texture to a 128x128 grid to perform fine spatial 
alignment between the images. Due to the limited 
rendering rate, this technique is only used for 
simulations with a high degree of fine spatial detail. 

Current Simulations and Future 
Directions 

A variety of models preceded the current state of 
the LSG design (Figure 2). A large assortment of 3-D 
tools and instrumentation are now available to replicate 
the precise objects used in the actual experiments (a few 
are shown in figure 3). For simulating advanced 



Figure 2: The Virtual GloveboX model 
environment with animal habitat module 
attached below and Space Station mounting 
rack behind. The 3-D models of equipment are 
created from hardware design models, 
photographs or from actual physical equipment. 



biomedical experiments in the VGX an anatomically 
based 3D model of a rat, using high resolution CT 
scans , was developed. Extensive development of the 
Spring simulation engine ’ ’ has demonstrated the 
ability to deform and cut simulated tissue in real-time 
using the PHANToM force-feedback devices, viewed 
from both a standard monitor and within the VGX using 
the geometry replication techniques. Additional 
simulation engines that perform only rigid body 
interactions have also been tested , allowing operational 
feasibility studies to be performed to determine 
equipment placement and management in a microgravity 
environment. 

Continued development of the VGX subsystems 
will increase the utility of this simulation system for 
enhancing the extent and quality of life sciences 
experiments conducted in space. Improvements in the 
fidelity of the registration of the visual world with the 
physical world could substantially increase the illusion 
of immersion in the workspace. The user interface for 
the VGX must be intuitive or it will not be useful as a 




Figure 3: Examples of tools and equipment 
available for the VGX: a) stereo-microscope, 
b) mass measuring device, c) plant 
pollination kit and d) lab tools. Objects are 
not to scale. 


realistic training tool. The current interface, using off- 
the-shelf virtual reality tracking and haptic feedback 
systems has cumbersome elements such as wired gloves, 
tethered magnetic trackers and haptic armatures inside 
the workspace. Further research will help reduce the 
need for encumbered interface devices and increase the 
value of the VGX as a virtual environment training tool. 
Finally, the graphics processing capabilities of the 
display subsystem must be utilized to a greater degree to 
include photo-realistic texturing, bump map lighting, 
and other advancements at the graphic hardware level to 
enrich the visual environment and reinforce the illusion 
of immersion. 

Conclusions 

The Virtual Glovebox is a virtual environment 
application designed to provide a realistic Life Sciences 
Glovebox training experience for astronauts that 
simulates the scientific experiments that will be 
conducted aboard the International Space Station. The 
Virtual Glovebox provides a high-fidelity virtual 
environment training tools by combining high resolution 
computer graphics with magnetic tracking and haptic 
feedback devices in an immersive desktop environment. 
The distributed design of the VGX system provides the 
benefits of a widely distributed multi-user system, 
resource conservation to improve processing speeds, and 
techniques for accessing the most recent advances in PC 
graphics while retaining a simulation engine on a high 
end multi-processor workstation. This provides a high 
level of realism to a user who can intuitively perform 
experimental tasks in a real-time microgravity or 
nominal 1-g simulation. Determining operational 
efficiencies for LSG equipment, optimal experimental 
design and simulation for training and evaluating 
operator performance are all VGX applications that will 
streamline processes associated with performing biology 
research in space. In this way, the NASA VGX 
advances existing LSG experiment planning, operations, 
training and trouble-shooting methods for astronauts and 
support crews, thus, helping to maximize the scientific 
return gained from future biological research in space. 
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